Electronic device and method for controlling the same

ABSTRACT

An electronic device according to an embodiment of the present disclosure may include a first biometric sensor configured to sense first biometric information; a second biometric sensor configured to sense second biometric information; a security module configured to set a multidimensional variable determination criterion including context information at the time of sensing at least one of the first biometric information and the second biometric information, and perform first user authentication by applying the multidimensional variable determination criterion to a matching score for the first biometric information or the second biometric information, and determine whether or not to perform second user authentication using the remaining biometric information based on a result of the first user authentication; and a controller configured to control an operation of the electronic device based on at least one of the first and second user authentication results carried out by the security module.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority under 35 U.S.C. § 119 to KoreanApplication No. 10-2018-0052750, filed on May 8, 2018, and No.10-2018-0091460, filed on Aug. 6, 2018, whose entire disclosures areherein incorporated by reference.

FIELD

The present disclosure relates to an electronic device capable ofperforming multimodal biometric authentication.

BACKGROUND

With the development of technology, the functions of electronic devicesare diversified. For example, the functions may include data and voicecommunication, photographing and video shooting through a camera, voicerecording, playing a music file through a speaker system, and displayingan image or video on a display unit. Some electronic devices furtherinclude an electronic game play function or perform a multimedia playerfunction. In particular, in recent years, electronic devices may receivemulticast signals that provide visual content such as broadcast, videoor television programs.

As it becomes multifunctional, an electronic device may be allowed tocapture still images or moving images, play music or video files, playgames, receive broadcast and the like, so as to be implemented as anintegrated multimedia player.

In order to support and enhance the functions of the electronic device,the improvement of structural or software elements of the terminal maybe taken into consideration.

As the functions of electronic devices are diversified, the importanceof user authentication for security enhancement has increased. Variousmethods such as a password method, a pattern method, and a biometricmethod may be used for user authentication.

As a method of security authentication, biometrics is a technology thatperforms user authentication using unique physical characteristics suchas a user's fingerprint, face, voice, iris, retina, blood vessels, orthe like. Such biometrics technology is less susceptible to theft orimitation, and is highly usable.

In recent years, in order to enhance security, technological advanceshave been made from unimodal biometrics that performs biometricauthentication using a single biometric information to multimodalbiometrics that performs biometric authentication using a plurality ofbiometric information.

In implementing such multimodal biometric authentication technology, itis required to develop various methods that utilize a plurality ofbiometric information. In the present disclosure, a method of performingbiometric authentication using a plurality of biometric information isproposed.

SUMMARY

An object of the present disclosure is to provide an electronic devicecapable of performing multimodal biometric authentication according to adetermination criterion modified according to a context in considerationof various environmental factors at the time of performing multimodalbiometric authentication, and a control method thereof.

Furthermore, another object of the present disclosure is to improve theauthentication accuracy and usability of multimodal biometricauthentication at the same time.

An electronic device according to an embodiment of the presentdisclosure, as an electronic device for performing multimodal biometricauthentication, may include a first biometric sensor configured to sensefirst biometric information; a second biometric sensor configured tosense second biometric information; a security module configured to seta multidimensional variable determination criterion including contextinformation at the time of sensing at least one of the first biometricinformation and the second biometric information, and perform first userauthentication by applying the multidimensional variable determinationcriterion to a matching score for the first biometric information or thesecond biometric information, and determine whether or not to performsecond user authentication using the remaining biometric informationbased on a result of the first user authentication; and a controllerconfigured to control an operation of the electronic device based on atleast one of the first and second user authentication results carriedout by the security module.

Furthermore, according to an embodiment, the multidimensional variabledetermination criterion may be determined in consideration of biometricsensors for sensing biometric information, the context information, thematching score, and a fake relevance score of the biometric informationas a whole.

Furthermore, according to an embodiment, the context information mayinclude at least one of ambient illuminance, ambient noise, ambienttemperature, characteristic information of biometric information,movement information, a posture of the electronic device, an executionof a preset application, a battery remaining amount of the electronicdevice, an input sequence of biometric information, and userinformation, and at least one of the context information may be selectedbased on the characteristics of biometric information.

Furthermore, according to an embodiment, the security module maycalculate a matching score by comparing the first biometric informationor the second biometric information with previously registered userinformation, and calculate a final score by combining the calculatedmatching score with a fake relevance score of the biometric information,and perform user authentication by applying a variable decision functiongenerated on the basis of the context information and the fake relevancescore of the biometric information to the calculated final matchingscore.

Furthermore, according to an embodiment, the fake relevance score of thebiometric information may include at least one of an anti-spoofing scoreand a quality score, and the variable decision function may be generatedby combining at least one of the anti-spoofing score and the qualityscore with the context information.

Furthermore, according to an embodiment, the variable decision functionmay be generated such that an upper threshold value and a lowerthreshold value related to the user authentication determination varyaccording to a level of the context information.

Furthermore, according to an embodiment, the upper threshold value maybe a value at which a false acceptance rate (FAR) becomes zero, thelower threshold value may be a value at which a false rejection rate(FRR) becomes zero, and at least one of the upper threshold value andthe lower threshold value may be increased or decreased according to alevel of the context information.

Furthermore, according to an embodiment, the second user authenticationmay be carried out when the authentication result of the first userauthentication does not correspond to both an authentication success andan authentication failure.

Furthermore, according to an embodiment, the no decision may be a casewhere a final score in consideration of the matching score, the fakerelevance score of the biometric information, and the contextinformation as a whole is less than the lower threshold value, and thelower threshold value may be varied according to the contextinformation.

Furthermore, according to an embodiment, when performing the second userauthentication, biometric information and context information for thesecond user authentication may be selected based on biometricinformation and context information used for the first userauthentication.

Furthermore, according to an embodiment, when performing the second userauthentication, both the first and second biometric information may beused, and a multidimensional variable determination criterion during thesecond user authentication may be determined by further considering amultidimensional variable determination criterion used for the firstuser authentication.

Furthermore, according to an embodiment, when performing the second userauthentication, a multidimensional variable determination criterionduring the second user authentication may be determined by furtherconsidering both a multidimensional variable determination criterionused for the first user authentication and context information used forthe first user authentication.

An operation method of an electronic device according to an embodimentof the present disclosure, as an operation method of an electronicdevice that performs multimodal biometric authentication, may includesensing biometric information of at least one of a first biometricsensor and a second biometric sensor; setting a multidimensionalvariable determination criterion including context information at thetime of sensing at least one of biometric information of the first andsecond biometric sensors; performing first user authentication byapplying the multidimensional variable determination criterion to amatching score for the at least one biometric information; determiningwhether or not to perform second user authentication using the remainingbiometric information based on a result of the first userauthentication; and controlling an operation of the electronic devicebased on at least one of the first and second user authenticationresults.

Furthermore, according to an embodiment, said setting a multidimensionalvariable determination criterion may be setting in consideration ofbiometric sensors for sensing biometric information, the contextinformation, the matching score, and a fake relevance score of thebiometric information as a whole.

Furthermore, according to an embodiment, said performing first userauthentication may include calculating a matching score by comparing thefirst biometric information or the second biometric information withpreviously registered user information; calculating a final score bycombining the calculated matching score with a fake relevance score ofbiometric information; and performing the first user authentication byapplying a variable decision function generated based on the contextinformation and the fake relevance score of biometric information to thecalculated final score.

Furthermore, according to an embodiment, the variable decision functionmay be generated such that an upper threshold value and a lowerthreshold value related to the user authentication determination varyaccording to a level of the context information, and the second userauthentication may be carried out when the final score corresponds tobetween variable upper and a lower threshold values as a result ofperforming the first user authentication.

An electronic device according to the present disclosure may determine avariable determination criterion in consideration of a fake relevancescore related to a surrounding environment at the time of performingbiometric authentication and the characteristics of biometricauthentication to perform biometric authentication, thereby improvingall the usability, convenience and sensing accuracy of biometricauthentication. In addition, whether or not to perform secondaryauthentication may be determined according to the execution result ofprimary authentication, thereby improving the authentication speed ofbiometric authentication. Furthermore, a multidimensional function inwhich a threshold value or a threshold range of no decision is variedeach time may be applied to authentication in consideration of variouscontext information collected while sensing biometric information and afake relevance score as a whole, thereby improving the accuracy ofbiometric authentication during secondary authentication.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments will be described in detail with reference to thefollowing drawings in which like reference numerals refer to likeelements wherein:

FIG. 1 is a block diagram for explaining an electronic device related tothe present disclosure;

FIG. 2 is a conceptual view illustrating a single biometricauthentication method;

FIGS. 3A through 3D are conceptual views illustrating a multimodalbiometric authentication method;

FIGS. 4A and 4B are graphs related to an error rate of a biometricauthentication determination;

FIG. 5 is a conceptual view showing a method of performing serialbiometric authentication during multimodal biometric authentication inthe related art;

FIGS. 6 and 7 are conceptual views showing a method of performing serialbiometric authentication;

FIGS. 8A through 8C are exemplary views for explaining a method ofapplying a variable determination criterion according to a context toperform serial biometric authentication during multimodal biometricauthentication according to the present disclosure;

FIGS. 9A through 9C are exemplary graphs for comparing an authenticationresult according to a determination criterion in the related art with anauthentication result according to a variable determination criterionaccording to the present disclosure during multimodal biometricauthentication according to the present disclosure;

FIG. 10 is a flowchart for explaining a method of applying a variabledetermination criterion according to a context to perform serialbiometric authentication during multimodal biometric authenticationaccording to the present disclosure;

FIGS. 11A through 14 are different examples showing specific multimodalbiometric authentication elements and context information to which avariable determination criterion is applied during multimodal biometricauthentication according to the present disclosure; and

FIG. 15 is a conceptual view illustrating a method of applying avariable determination criterion according to a context and fusing aplurality of sensors, context information, and the variabledetermination criterion to perform parallel biometric authenticationduring multimodal biometric authentication according to the presentdisclosure.

DETAILED DESCRIPTION

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the accompanying drawings, and thesame or similar elements are designated with the same numeral referencesregardless of the numerals in the drawings and their redundantdescription will be omitted. A suffix “module” or “unit” used forconstituent elements disclosed in the following description is merelyintended for easy description of the specification, and the suffixitself does not give any special meaning or function. In describing thepresent disclosure, moreover, the detailed description will be omittedwhen a specific description for publicly known technologies to which theinvention pertains is judged to obscure the gist of the presentdisclosure. The accompanying drawings are used to help easily understandthe technical idea of the present disclosure and it should be understoodthat the idea of the present disclosure is not limited by theaccompanying drawings.

It will be understood that although the terms first, second, etc. may beused herein to describe various elements, these elements should not belimited by these terms. These terms are generally only used todistinguish one element from another.

It will be understood that when an element is referred to as being“connected with” another element, the element can be directly connectedwith the other element or intervening elements may also be present. Onthe contrary, in case where an element is “directly connected” or“directly linked” to another element, it should be understood that anyother element is not existed therebetween.

A singular representation may include a plural representation as far asit represents a definitely different meaning from the context.

Terms “include” or “has” used herein should be understood that they areintended to indicate an existence of several components or severalsteps, disclosed in the specification, and it may also be understoodthat part of the components or steps may not be included or additionalcomponents or steps may further be included.

Portable electronic devices described herein may include cellularphones, smart phones, laptop computers, digital broadcasting terminals,personal digital assistants (PDAs), portable multimedia players (PMPs),navigators, slate PCs, tablet PCs, ultrabooks, wearable devices (forexample, smart watches, smart glasses, head mounted displays (HMDs)),smart vehicles and the like.

However, it may be easily understood by those skilled in the art thatthe configuration according to the exemplary embodiments of thisspecification can also be applied to stationary terminals such asdigital TV, desktop computers and the like, excluding a case of beingapplicable only to the electronic devices.

FIG. 1 is a block diagram for explaining an electronic device related tothe present disclosure.

Referring to FIG. 1, the electronic device may include an input unit120, a sensing unit 140, an output unit 150, an interface unit 160, amemory 170, a controller 180, a security module 181, and a power supplyunit 190, and the like. The components shown in FIG. 1 are not essentialfor implementing an electronic device, and thus the electronic devicedescribed herein may have more or fewer components than those listedabove.

In more detail, the wireless communication unit 110 of those componentsmay typically include one or more modules which permit wirelesscommunications between the electronic device 100 and a wirelesscommunication system, between the electronic device 100 and anotherelectronic device 100, or between the electronic device 100 and anexternal server. In addition, the wireless communication unit 110 mayinclude one or more modules for connecting the electronic device 100 toone or more networks.

The wireless communication unit 110 may include at least one of abroadcast receiving module 111, a mobile communication module 112, awireless Internet module 113, a short-range communication module 114, alocation information module 115 and the like.

The input unit 120 may include a camera 121 for inputting an imagesignal, a microphone 122 or an audio input module for inputting an audiosignal, or a user input unit 123 (for example, a touch key, a push key(or a mechanical key), etc.) for allowing a user to input information.Audio data or image data collected by the input unit 120 may be analyzedand processed by a user's control command.

The sensing unit 140 may include at least one sensor which senses atleast one of information within the electronic device, surroundingenvironment information of the electronic device, and user information.For example, the sensing unit 140 may include a proximity sensor 141, anillumination sensor 142, a touch sensor, an acceleration sensor, amagnetic sensor, a G-sensor, a gyroscope sensor, a motion sensor, an RGBsensor, an infrared (IR) sensor, a finger scan sensor, a ultrasonicsensor, an optical sensor (for example, refer to the camera 121), amicrophone 122, a battery gage, an environment sensor (for example, abarometer, a hygrometer, a thermometer, a radiation detection sensor, athermal sensor, a gas sensor, etc.), and a chemical sensor (for example,an electronic nose, a health care sensor, a biometric sensor, etc.). Thebiometric sensor 143 may include an iris sensor, a face recognitionsensor, a PPG sensor, a voice sensor, and the like. On the other hand,the electronic device 100 disclosed herein may be configured to utilizeinformation obtained from sensing unit 140, and in particular,information obtained from one or more sensors of the sensing unit 140,and combinations thereof.

The output unit 150 may be configured to output an audio signal, a videosignal or a tactile signal. The output unit 150 may include a displayunit 151, an audio output module 152, a haptic module 153, an opticaloutput unit 154 and the like. The display unit 151 may have aninter-layered structure or an integrated structure with a touch sensorin order to facilitate a touch screen. The touch screen may provide anoutput interface between the electronic device 100 and a user, as wellas functioning as the user input unit 123 which provides an inputinterface between the electronic device 100 and the user.

The interface unit 160 may serve as an interface with various types ofexternal devices connected with the electronic device 100. The interfaceunit 160, for example, may include wired or wireless headset ports,external power supply ports, wired or wireless data ports, memory cardports, ports for connecting a device having an identification module,audio input/output (I/O) ports, video I/O ports, earphone ports, or thelike. The electronic device 100 may execute an appropriate controlassociated with a connected external device, in response to the externaldevice being connected to the interface unit 160.

In addition, the memory 170 stores data that support various functionsof the electronic device 100. The memory 170 is typically implemented tostore data to support various functions or features of the electronicdevice 100. For instance, the memory 170 may be configured to storeapplication programs executed in the electronic device 100, data orinstructions for operations of the electronic device 100, and the like.At least some of those application programs may be downloaded from anexternal server via wireless communication. Some others of thoseapplication programs may be installed within the electronic device 100at the time of being shipped for basic functions of the electronicdevice 100 (for example, receiving a call, placing a call, receiving amessage, sending a message, etc.). On the other hand, the applicationprograms may be stored in the memory 170, installed in the electronicdevice 100, and executed by the controller 180 to perform an operation(or a function) of the electronic device 100.

The controller 180 may typically control an overall operation of theelectronic device 100 in addition to the operations associated with theapplication programs. The controller 180 may provide or processinformation or functions appropriate for a user by processing signals,data, information and the like, which are input or output by the variouscomponents depicted in FIG. 1A, or activating application programsstored in the memory 170.

Furthermore, the controller 180 may control at least part of thecomponents illustrated in FIG. 1A, in order to drive the applicationprograms stored in the memory 170. In addition, the controller 180 maydrive the application programs by combining at least two of thecomponents included in the electronic device 100 for operation.

The power supply unit 190 may receive external power or internal powerand supply appropriate power required for operating respective elementsand components included in the electronic device 100 under the controlof the controller 180. The power supply unit 190 may include a battery,and the battery may be an embedded battery or a replaceable battery.

At least part of those elements and components may be combined toimplement operation and control of the terminal or a control method ofthe electronic device according to various exemplary embodimentsdescribed herein. Furthermore, the operation and control or the controlmethod of the portable electronic device may be implemented in theportable electronic device in such a manner of activating at least oneapplication program stored in the memory 170.

Hereinafter, each aforementioned component will be described in moredetail with reference to FIG. 1A, prior to explaining various exemplaryembodiments implemented by the electronic device 100 having theconfiguration.

First, the wireless communication unit 110 will be described. Thebroadcast receiving module 111 of the wireless communication unit 110may receive a broadcast signal and/or broadcast associated informationfrom an external broadcast managing entity via a broadcast channel. Thebroadcast channel may include a satellite channel and/or a terrestrialchannel. At least two broadcast receiving modules 111 may be provided inthe portable electronic device 100 to simultaneously receive at leasttwo broadcast channels or switch the broadcast channels.

The mobile communication module 112 may transmit/receive wirelesssignals to/from at least one of network entities, for example, a basestation, an external terminal, a server, and the like, on a mobilecommunication network, which is constructed according to technicalstandards or transmission methods for mobile communications (forexample, Global System for Mobile communication (GSM), Code DivisionMulti Access (CDMA), Code Division Multi Access 2000 (CDMA2000),Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO),Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), HighSpeed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long TermEvolution-Advanced (LTE-A), etc.)

Here, the wireless signals may include audio call signal, video(telephony) call signal, or various formats of data according totransmission/reception of text/multimedia messages.

The wireless Internet module 113 means a module for supporting wirelessInternet access. The wireless Internet module 113 may be built-in orexternally installed to the electronic device 100. The wireless Internetmodule 113 may transmit and/or receive wireless signals viacommunication networks according to wireless Internet technologies.

Examples of such wireless Internet access may include Wireless LAN(WLAN), Wireless-Fidelity (Wi-Fi), Wireless Fidelity Direct (Wi-FiDirect), Digital Living Network Alliance (DLNA), Wireless Broadband(WiBro), World Interoperability for Microwave Access (WiMAX), High SpeedDownlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA),LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced), and thelike. The wireless Internet module 113 may transmit/receive dataaccording to at least one wireless Internet technology within a rangeincluding even Internet technologies which are not aforementioned.

From the perspective that the wireless Internet accesses according toWibro, HSDPA, GSM, CDMA, WCDMA, LTE, LTE-A and the like are executed viaa mobile communication network, the wireless Internet module 113 whichperforms the wireless Internet access via the mobile communicationnetwork may be understood as a type of the mobile communication module112.

The short-range communication module 114 denotes a module forshort-range communications. Suitable technologies for implementing theshort-range communications may include BLUETOOTH™, Radio FrequencyIDentification (RFID), Infrared Data Association (IrDA), Ultra-WideBand(UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity(Wi-Fi), Wi-Fi Direct, and the like. The short-range communicationmodule 114 may support wireless communications between the electronicdevice 100 and a wireless communication system, between the electronicdevice 100 and another electronic device 100, or between the electronicdevice and a network where another electronic device (or an externalserver) is located, via wireless personal area networks. The short-rangecommunication module 114 denotes a module for short-rangecommunications.

In some embodiments, another electronic device (which may be configuredsimilarly to electronic device 100) may be a wearable device, forexample, a smart watch, a smart glass or a head mounted display (HMD),which is able to exchange data with the electronic device 100 (orotherwise cooperate with the electronic device 100). The short-rangecommunication module 114 may sense (recognize) a wearable device, whichis able to communicate with the electronic device 100, near theelectronic device 100. In addition, when the sensed wearable device is adevice which is authenticated to communicate with the electronic device100 according to the present disclosure, the controller 180 may transmitat least part of data processed in the electronic device 100 to thewearable device via the short-range communication module 114. Hence, auser of the wearable device may use the data processed in the electronicdevice 100 on the wearable device. For example, when a call is receivedin the electronic device 100, the user may answer the call using thewearable device. Also, when a message is received in the electronicdevice 100, the user can check the received message using the wearabledevice.

The location information module 115 is generally configured to detect,calculate, derive or otherwise identify a position of the electronicdevice. As an example, the location information module 115 includes aGlobal Position System (GPS) module, a WiFi module, or both. Forexample, when the electronic device uses the GPS module, a position ofthe electronic device may be acquired using a signal sent from a GPSsatellite. As another example, when the electronic device uses the Wi-Fimodule, a position of the electronic device may be acquired based oninformation associated with a wireless access point (AP) which transmitsor receives a wireless signal to or from the Wi-Fi module. According tothe need, the location information module 115 may perform any functionof the other modules of the wireless communication unit 110 to obtaindata on the location of the electronic device. As a module used toacquire the location (or current location) of the electronic device, thelocation information module 115 may not be necessarily limited to amodule for directly calculating or acquiring the location of theelectronic device.

The input unit 120 may be configured to provide an audio or video signal(or information) input to the electronic device or information input bya user to the electronic device. For the input of the audio information,the electronic device 100 may include one or a plurality of cameras 121.The camera 121 processes a image frame, such as still picture or video,acquired by an image sensor in a video phone call or image capturingmode. The processed image frames may be displayed on the display unit151. On the other hand, the plurality of cameras 121 disposed in theelectronic device 100 may be arranged in a matrix configuration. By useof the cameras 121 having the matrix configuration, a plurality of imageinformation having various angles or focal points may be input into theelectronic device 100. As another example, the cameras 121 may belocated in a stereoscopic arrangement to acquire left and right imagesfor implementing a stereoscopic image.

The microphone 122 may process an external audio signal into electricaudio data. The processed audio data may be utilized in various mannersaccording to a function being executed in the electronic device 100 (oran application program being executed). On the other hand, themicrophone 122 may include assorted noise removing algorithms to removenoise generated in the course of receiving the external audio signal.

The user input unit 123 may receive information input by a user. Wheninformation is input through the user input unit 123, the controller 180may control an operation of the electronic device 100 to correspond tothe input information. The user input unit 123 may include a mechanicalinput element (or a mechanical key, for example, a button, a domeswitch, a jog wheel, a jog switch or the like located on a front/rearsurface or a side surface of the electronic device 100), and atouch-sensitive input element. As one example, the touch-sensitive inputmeans may be a virtual key, a soft key or a visual key, which isdisplayed on a touch screen through software processing, or a touch keywhich is disposed on a portion except for the touch screen. On the otherhand, the virtual key or the visual key may be displayable on the touchscreen in various shapes, for example, graphic, text, icon, video or acombination thereof.

On the other hand, the sensing unit 140 may sense at least one ofinternal information of the electronic device, surrounding environmentinformation of the electronic device and user information, and generatea sensing signal corresponding thereto. The controller 180 may controlan operation of the electronic device 100 or execute data processing, afunction or an operation associated with an application programinstalled in the electronic device 100 based on the sensing signal.Hereinafter, description will be given in more detail of representativesensors of various sensors which may be included in the sensing unit140.

First, a proximity sensor 141 refers to a sensor to sense presence orabsence of an object approaching to a surface to be sensed, or an objectdisposed near a surface to be sensed, by using an electromagnetic fieldor infrared rays without a mechanical contact. The proximity sensor 141may be arranged at an inner region of the electronic device covered bythe touch screen, or near the touch screen.

The proximity sensor 141, for example, may include any of a transmissivetype photoelectric sensor, a direct reflective type photoelectricsensor, a mirror reflective type photoelectric sensor, a high-frequencyoscillation proximity sensor, a capacitance type proximity sensor, amagnetic type proximity sensor, an infrared rays proximity sensor, andthe like. When the touch screen is implemented as a capacitance type,the proximity sensor 141 may sense proximity of a pointer to the touchscreen by changes of an electromagnetic field, which is responsive to anapproach of an object with conductivity. In this case, the touch screen(touch sensor) may also be categorized as a proximity sensor.

On the other hand, for the sake of brief explanation, a behavior inwhich the pointer is positioned to be proximate onto the touch screenwithout contact will be referred to as “proximity touch,” whereas abehavior in which the pointer substantially comes into contact with thetouch screen will be referred to as “contact touch.” For the positioncorresponding to the proximity touch of the pointer on the touch screen,such position will correspond to a position where the pointer facesperpendicular to the touch screen upon the proximity touch of thepointer. The proximity sensor 141 may sense proximity touch, andproximity touch patterns (e.g., distance, direction, speed, time,position, moving state, etc.). On the other hand, the controller 180 mayprocess data (or information) corresponding to the proximity touches andthe proximity touch patterns sensed by the proximity sensor 141, andoutput visual information corresponding to the process data on the touchscreen. In addition, the controller 180 may control the electronicdevice 100 to execute different operations or process different data (orinformation) according to whether a touch with respect to the same pointon the touch screen is either a proximity touch or a contact touch.

A touch sensor may sense a touch (or touch input) applied onto the touchscreen (or the display unit 151) using at least one of various types oftouch methods, such as a resistive type, a capacitive type, an infraredtype, a magnetic field type, and the like.

As one example, the touch sensor may be configured to convert changes ofpressure applied to a specific part of the display unit 151 or acapacitance occurring from a specific part of the display unit 151, intoelectric input signals. Also, the touch sensor may be configured tosense not only a touched position and a touched area, but also touchpressure. Here, the touch object body may be a finger, a touch pen orstylus pen, a pointer, or the like as an object through which a touch isapplied to the touch sensor.

When a touch input is sensed by a touch sensor, corresponding signalsmay be transmitted to a touch controller. The touch controller mayprocess the received signals, and then transmit corresponding data tothe controller 180. Accordingly, the controller 180 may sense whichregion of the display unit 151 has been touched. Here, the touchcontroller may be a component separate from the controller 180 or thecontroller 180 itself.

On the other hand, the controller 180 may execute a different control orthe same control according to a type of an object which touches thetouch screen (or a touch key provided in addition to the touch screen).Whether to execute the different control or the same control accordingto the object which gives a touch input may be decided based on acurrent operating state of the electronic device 100 or a currentlyexecuted application program.

Meanwhile, the touch sensor and the proximity sensor may be executedindividually or in combination, to sense various types of touches, suchas a short (or tap) touch, a long touch, a multi-touch, a drag touch, aflick touch, a pinch-in touch, a pinch-out touch, a swype touch, ahovering touch, and the like.

An ultrasonic sensor may be configured to recognize position informationrelating to a sensing object by using ultrasonic waves. On the otherhands, the controller 180 may calculate a position of a wave generationsource based on information sensed by an illumination sensor and aplurality of ultrasonic sensors. Since light is much faster thanultrasonic waves, a time for which the light reaches the optical sensormay be much shorter than a time for which the ultrasonic wave reachesthe ultrasonic sensor. The position of the wave generation source may becalculated using the fact. In more detail, the position of the wavegeneration source may be calculated by using a time difference from thetime that the ultrasonic wave reaches based on the light as a referencesignal.

The camera 121 constructing the input unit 120 may be a type of camerasensor. The camera sensor may include at least one of a photo sensor (orimage sensor) and a laser sensor.

Implementing the camera 121 with a laser sensor may allow detection of atouch of a physical object with respect to a 3D stereoscopic image. Thephoto sensor may be laminated on the display device. The photo sensormay be configured to scan a movement of the sensing object in proximityto the touch screen. In more detail, the photo sensor may include photodiodes and transistors at rows and columns to scan content placed on thephoto sensor by using an electrical signal which changes according tothe quantity of applied light. Namely, the photo sensor may calculatethe coordinates of the sensing object according to variation of light tothus obtain position information of the sensing object.

The display unit 151 may display (output) information processed in theelectronic device 100. For example, the display unit 151 may displayexecution screen information of an application program driven in theelectronic device 100 or user interface (UI) and graphic user interface(GUI) information in response to the execution screen information.

Furthermore, the display unit 151 may also be implemented as astereoscopic display unit for displaying stereoscopic images.

The stereoscopic display unit may employ a stereoscopic display schemesuch as stereoscopic scheme (a glass scheme), an auto-stereoscopicscheme (glassless scheme), a projection scheme (holographic scheme), orthe like.

The audio output module 152 is generally configured to output audiodata. Such audio data may be obtained from any of a number of differentsources, such that the audio data may be received from the wirelesscommunication unit 110 or may have been stored in the memory 170. Also,the audio output module 152 may also provide audible output signalsassociated with a particular function (e.g., a call signal receptionsound, a message reception sound, etc.) carried out by the electronicdevice 100. The audio output module 152 may include a receiver, aspeaker, a buzzer or the like.

A haptic module 153 may generate various tactile effects the that usermay feel. A typical example of the tactile effect generated by thehaptic module 153 may be vibration. Strength, pattern and the like ofthe vibration generated by the haptic module 153 may be controllable bya user selection or setting of the controller. For example, the hapticmodule 153 may output different vibrations in a combining manner or asequential manner.

Besides vibration, the haptic module 153 may generate various othertactile effects, including an effect by stimulation such as a pinarrangement vertically moving with respect to a contact skin, a sprayforce or suction force of air through a jet orifice or a suctionopening, a touch on the skin, a contact of an electrode, electrostaticforce, etc., an effect by reproducing the sense of cold and warmth usingan element that can absorb or generate heat, and the like.

The haptic module 153 may be configured to transmit tactile effectsthrough a user's direct contact, or a user's muscular sense using afinger or a hand. The haptic module 153 may be implemented in two ormore in number according to the configuration of the electronic device100.

An optical output module 154 may output a signal for indicating an eventgeneration using the light of a light source of the electronic device100. Examples of events generated in the electronic device 100 mayinclude a message reception, a call signal reception, a missed call, analarm, a schedule notice, an email reception, an information receptionthrough an application, and the like.

A signal output by the optical output module 154 may be implemented insuch a manner that the electronic device emits monochromatic light orlight with a plurality of colors. The signal output may be terminated asthe electronic device senses a user's event checking.

The interface unit 160 serves as an interface for external devices to beconnected with the electronic device 100. For example, the interfaceunit 160 can receive data transmitted from an external device, receivepower to transfer to elements and components within the electronicdevice 100, or transmit internal data of the electronic device 100 tosuch external device. The interface unit 160 may include wired orwireless headset ports, external power supply ports, wired or wirelessdata ports, memory card ports, ports for connecting a device having anidentification module, audio input/output (I/O) ports, video I/O ports,earphone ports, or the like.

The identification module may be a chip that stores various informationfor authenticating authority of using the electronic device 100 and mayinclude a user identity module (UIM), a subscriber identity module(SIM), a universal subscriber identity module (USIM), and the like. Inaddition, the device having the identification module (also referred toherein as an “identification device”) may take the form of a smart card.Accordingly, the identifying device may be connected with the electronicdevice 100 via the interface unit 160.

Also, the interface unit 160 may serve as a path for power to besupplied from an external cradle to the electronic device 100 when theelectronic device 100 is connected to the external cradle or as a pathfor transferring various command signals inputted from the cradle by auser to the electronic device 100. Such various command signals or powerinputted from the cradle may operate as signals for recognizing that theelectronic device 100 has accurately been mounted to the cradle.

The memory 170 can store programs to support operations of thecontroller 180 and store input/output data (for example, phonebook,messages, still images, videos, etc.). The memory 170 may store dataassociated with various patterns of vibrations and audio which areoutput in response to touch inputs on the touch screen.

The memory 170 may include at least one type of storage medium includinga Flash memory, a hard disk, a multimedia card micro type, a card-typememory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), aStatic Random Access Memory (SRAM), a Read-Only Memory (ROM), anElectrically Erasable Programmable Read-Only Memory (EEPROM), aProgrammable Read-Only memory (PROM), a magnetic memory, a magneticdisk, and an optical disk. Also, the electronic device 100 may operate aweb storage which performs the storage function of the memory 170 on theInternet.

As aforementioned, the controller 180 may typically control the generaloperations of the electronic device 100. For example, the controller 180may set or release a lock state for restricting a user from inputting acontrol command with respect to applications when a state of theelectronic device meets a preset condition.

Furthermore, the controller 180 may also perform controlling andprocessing associated with voice calls, data communications, videocalls, and the like, or perform pattern recognition processing torecognize a handwriting input or a picture drawing input performed onthe touch screen as characters or images, respectively. In addition, thecontroller 180 may control one or combination of those components inorder to implement various exemplary embodiment disclosed herein on theelectronic device 100.

The security module 181 controls an operation related security among theoperations of the electronic device. For example, when a biometricauthentication function is executed, the security module 181 may performcontrol related to biometric authentication. For example, the securitymodule 181 may perform biometric authentication using an artificialneural network algorithm or an SVM algorithm, which is an algorithm forbiometric authentication. In addition, the security module 181 mayperform an algorithmic operation, such as Fuzzy logic, Dempster-Shafertheory, SVM, relevance vector machine (RVM) mean rule, Monte Carloapproach, phase stretch transform (PST), neural network, principalcomponent analysis, Fisherfaces, Wavelet and Elastic Matching, or thelike, which is an algorithm for biometric authentication.

The security module 181 may communicate with the controller 180 totransmit and receive data, thereby controlling an overall operation ofthe electronic device. For example, the controller 180 may receive userauthentication result data from the security module 181 and control anoperation of the electronic device based on the received data. Inaddition, the security module 181 may receive a control command forperforming biometric authentication from the controller 180, therebyperforming biometric authentication.

On the other hand, in FIG. 1, the security module 181 and the controller180 are illustrated as being separate components, but the presentdisclosure is not limited thereto, and the security module 181 may beconfigured as one component of the controller 180.

The power supply unit 190 may receive external power or internal powerand supply appropriate power required for operating respective elementsand components included in the electronic device 100 under the controlof the controller 180. The power supply unit 190 may include a battery,which is typically rechargeable or be detachably coupled to the terminalbody for charging.

Furthermore, the power supply unit 190 may include a connection port.The connection port may be configured as one example of the interfaceunit 160 to which an external (re)charger for supplying power torecharge the battery is electrically connected.

As another example, the power supply unit 190 may be configured torecharge the battery in a wireless manner without use of the connectionport. Here, the power supply unit 190 may receive power, transferredfrom an external wireless power transmitter, using at least one of aninductive coupling method which is based on magnetic induction or amagnetic resonance coupling method which is based on electromagneticresonance.

Various embodiments described herein may be implemented in acomputer-readable or its similar medium using, for example, software,hardware, or any combination thereof.

Hereinafter, a method of performing biometric authentication in theelectronic device described in FIG. 1 will be described in detail. FIG.2 is a conceptual view illustrating a single biometric authenticationmethod.

Referring to FIG. 2, single biometric authentication may include thesteps of acquisition 210, feature extraction 220, matching 230 anddecision 240.

In the acquisition step 210, biometric information may be acquiredthrough a biometric sensor. The biometric information may include auser's own biometric information such as fingerprint, face, voice, vein,iris, and the like.

In the feature extraction step 220, the features of the biometricinformation may be extracted. The feature is information capable ofrecognizing the unique characteristic of each person. For example, inthe case of a fingerprint, a point representing a specific shape of thefingerprint may be set as a feature. These features are set differentlyfor each biometric authentication method.

In the matching step 230, a matching score between previously registereduser information and sensed biometric information may be calculated. Thepreviously registered user information is biometric information storedin advance by a user prior to performing biometric authentication. Theuser store fingerprint information, face information, voice information,vein information, iris information, and the like in advance in thememory 170 in a templet shape.

The matching score indicates a similarity between the previouslyregistered user information and the biometric information. Variousalgorithms previously known in the related art may be used as analgorithm for calculating matching scores.

In the decision step 240, user authentication may be carried out usingthe matching score and the decision function. The decision function is afunction that determines whether a user who enters biometric informationis a genuine user or an impostor user. The decision function may be setto a specific threshold value, or may be set to a multidimensionalfunction.

The decision function may be set to an initial setting value (default)by a manufacturer of a biometric authentication function. Furthermore,the decision function may change the initial setting value using theuser's biometric information sensed through the biometric sensor.Accordingly, the electronic device may improve the speed and accuracy ofbiometric recognition as a lot of biometric operations are carried out.

In addition, the decision function may be generated differentlyaccording to information used to generate the decision function.Moreover, the differently generated decision function may be stored inthe memory 170 in a plurality of ways. For example, the decisionfunction may be generated with only a matching score, or may begenerated using a matching score and a spoofing score. In this case,both of the decision functions may be stored in the memory 170, andbiometric authentication may be carried out using any one of thedecision functions as needed.

A single biometric authentication method has been described above.Hereinafter, a multimodal biometric authentication method will bedescribed. FIGS. 3A through 3D are conceptual views illustrating amultimodal biometric authentication method.

Multimodal biometric authentication may be divided into four typesaccording to the time of fusioning a plurality of biometric information.Here, fusion refers to an operation of combining a plurality ofinformation according to a preset algorithm to generate one information,and may be used in terms of coupling, combination, fusion, and matching.

FIG. 3A has shown a sensor fusion method 310. The sensor fusion method310 is a method of combining a plurality of biometric informationacquired from different sensors in the step of acquiring biometricinformation. Specifically, the sensor fusion method is a method offusioning biometric information sensed by different biometric sensorsand extracting features from the fusioned information.

FIG. 3A has shown a feature fusion method 320. The feature fusion method320 is a method of respectively extracting feature from a plurality ofbiometric information acquired from different biometric sensors in thestep of extracting the features of biometric information, and combiningthe respectively extracted features.

FIG. 3A has shown a score fusion method 330. The score fusion method 330is a method of combining matching scores calculated for each of theplurality of biometric information in the step of matching biometricinformation.

FIG. 3A has shown a decision fusion method 340. The decision fusionmethod 340 is a method of combining decision results calculated for eachof the plurality of biometric information in the step of determiningbiometric information.

Various methods of multimodal biometric authentication have beendescribed above. Hereinafter, an error rate of biometrics related to adecision function will be described in the case of user authenticationthrough biometric authentication. FIGS. 4A and 4B are graphs related toan error rate of a biometric authentication determination.

Graph “a” in FIG. 4A is a graph showing a similarity distributionbetween the biometric information of a genuine user and the previouslyregistered user information at the time of biometric authentication, andgraph “b” in FIG. 4A is a graph showing a similarity distributionbetween the biometric information of an impostor user and the previouslyregistered user information.

The graphs “a” and “b” have overlapping portions, and the electronicdevice 100 determines a user as a genuine user when having a similarityhigher than a threshold value indicated by dotted line aa′, anddetermines the user as an impostor user when having a similarity lowerthan the threshold value. Here, the threshold value may be a valuedetermined by a provider providing a biometric authentication function,and denotes the above-described decision function.

On the other hand, a false rejection rate (FRR) illustrated in FIG. 4Aindicates a rate determined to be an impostor user although the user isa genuine user. Since the higher the FRR is, the higher the thresholdvalue, and thus a probability that the user who has entered biometricinformation is determined as a genuine user is decreased, the securityof the biometric authentication may be enhanced. Since the lower the FRRis, the lower the threshold value, and thus a probability that the userwho has entered biometric information is determined as a genuine user isincreased, the security of the biometric authentication may be reduced.

A false acceptance rate (FAR) indicates an error rate determined to be agenuine user although the user is a genuine user. FAR is a conceptcontrary to FRR, and since the higher the FAR is, the lower thethreshold value, and thus a probability that the user who has enteredbiometric information is determined as a genuine user is increased, thesecurity of the biometric authentication may be reduced.

FIG. 4B is a graph showing a relationship between a FRR and a FAR. TheFRR and the FAR may be inversely proportional to each other. A thresholdvalue corresponding to region d having a high FRR and a low FAR may beused for applications requiring high security although having a lowauthentication speed. For example, a threshold value for this area maybe set in a billing application, a banking application, which strictlydetermines a genuine user. On the contrary, a threshold valuecorresponding region c having a low FRR and a high FAR may be used forapplications requiring low security although having a highauthentication speed. For example, a threshold value corresponding tothis region may be used for an unlock function or the like. Thethreshold value (i.e., decision function) of the biometricauthentication function may be determined in consideration of a securitylevel of functions to be executed through biometric authentication.Parameters related to an error at the time of biometric authenticationhave been described above.

Hereinafter, an algorithm that can be used in a score fusion methodamong multimodal biometric authentication methods will be described.

In the score fusion method, a variety of artificial intelligencealgorithms that combine match scores may be used. A combination-basedscore fusion algorithm, a classifier-based score fusion algorithm, and adensity-based score fusion algorithm may be used in an algorithm thatcan be used in the score fusion method.

The combination-based score fusion algorithm may include statisticalrules, dynamic weighting, triangular norms, and the like. Theclassifier-based score fusion algorithm may include support vectormachine (SVM), AdaBoost (RS-ADA), and Dampster-Shafer (DS). Thedensity-based score fusion algorithm may include a likelihood feature(LF).

In addition, various algorithms and the like known in a secure fusionmethod may be used in the present disclosure, and the detaileddescription thereof will be omitted in order not to depart from thespirit of the present disclosure.

Hereinafter, biometric authentication in a serial manner in which two ormore different biometric authentication methods are sequentiallyperformed to perform biometric authentication will be described withreference to the drawings.

FIG. 5 is a conceptual view illustrating a method of performingbiometric authentication in a serial manner during multimodal biometricauthentication in the related art.

Biometric authentication may be classified according to biometricinformation. For example, the biometric authentication may include faceauthentication, fingerprint authentication, voice authentication, irisauthentication, vein authentication, and the like, and a user mayperform biometric authentication using a variety of human bodyinformation capable of exhibiting a person's unique characteristics.

The multimodal biometric authentication is a method of performingbiometric authentication using different biometric information.

The method of multimodal biometric authentication may include serialbiometric authentication and parallel biometric authentication accordingto the time of acquisition of biometric information. Specifically, theserial method is a method of sequentially acquiring a plurality ofbiometric information, and the parallel method is a method of acquiringa plurality of biometric information at the same time and fusing theacquired biometric information at the same time to perform biometricauthentication

The serial method is advantageous in that a period of time required forbiometric authentication is short because one method of biometricauthentication is carried out at a time, and has good usability.However, the serial method has a lower accuracy compared to the parallelmethod.

According to the present disclosure, a method of performing biometricauthentication using a serial method will be described. Hereinafter, aserial method in the related art will be described in more detail withreference to the drawings.

Referring to FIG. 5, serial biometric authentication acquires onebiometric information to perform biometric authentication (primary userauthentication, 510). For a result of performing the primary userauthentication 510, there are an authentication success 520 in whichpreviously registered user information and acquired biometricinformation match each other or an authentication failure 530 in whichpreviously registered user information and acquired biometricinformation do not match each other. In the case of the authenticationfailure 530, serial biometric authentication acquires another biometricinformation to perform additional authentication (secondary userauthentication, 540).

In other words, the serial biometric authentication sequentiallyrecognizes and authenticates different biometric information accordingto a preset order. Such a serial biometric authentication may also bereferred to as sequential authentication, cascaded authentication, ormulti-stage fusion authentication.

For the serial biometric authentication, optimization of thresholdsbased on linear model, a symmetric rejection method, a marcialis'smethod, a SPRT-based method, serial fusion based semi-supervisedlearning techniques, a quality-based adaptive context switchingalgorithm, or the like, may be used.

Hereinafter, serial multimodal biometric authentication will bedescribed in more detail. FIGS. 6 and 7 are conceptual views showing aspecific method of performing serial biometric authentication.

Referring to FIG. 6, the serial biometric authentication is divided intoa primary user authentication 610 and a secondary user authentication650. The primary user authentication 610 and the secondary userauthentication 650 are biometric authentication methods using differentbiometric information. For example, the primary user authentication 610may be fingerprint recognition, and the secondary user authentication650 may be face recognition.

The execution result of the primary user authentication 610 is dividedinto an authentication success 620, an authentication failure 630, andno decision 640.

Specifically, when a degree of similarity between previously registereduser information and acquired biometric information is greater than (orequal to) a first reference value (P1 in FIG. 7), the security module181, which processes biometric information, determines that it is anauthentication success 620.

The first reference value (P1) is a value at which the false acceptancerate (FAR) becomes zero. Here, the false acceptance rate (FAR) is a rateat which an impostor user is mistaken as a genuine user. The value atwhich the FAR becomes zero may be referred to as a threshold value atwhich an impostor user is not mistaken as a genuine user. Accordingly,if the similarity degree is a first reference value, that is, a value atwhich the FAR is greater than zero (or equal to or greater than zero),it will be determined as an authentication success 620.

Furthermore, when a degree of similarity between previously registereduser information and acquired biometric information is less than (orequal to) a second reference value (P2 in FIG. 7), the security module181 determines that it is an authentication failure 630.

The second reference value (P2) is a value at which the false rejectionrate (FRR) becomes zero. Here, the false rejection rate (FRR) is at rateat which a genuine user is mistaken as an impostor user. The value atwhich the FRR becomes zero may be referred to as a threshold value atwhich a genuine user is not mistaken as an impostor user. Accordingly,if the similarity degree is a second reference value, that is, a valueat which the FAR is less than zero (or equal to or less than zero), itwill be determined as an authentication failure 630.

The security module 181 may perform an authentication initializationoperation in the case of the authentication failure 630. Theauthentication initialization operation denotes an operation ofswitching to a standby state capable of performing the primary userauthentication again.

Furthermore, when a degree of similarity between previously registereduser information and acquired biometric information is greater than orequal to a second reference value (P2) but less than or equal to a firstreference value (P1), the security module 181 determines that it is nodecision 640. In this case, the security module 181 performs thesecondary user authentication 650. In other words, the secondary userauthentication 650 may be carried out when it is no decision duringserial multimodal biometric authentication.

During the serial biometric authentication, the secondary userauthentication 650 may perform biometric authentication using biometricinformation acquired from the primary user authentication and newlyacquired biometric information. At this time, the newly acquiredbiometric information is biometric information different from thebiometric information acquired from the primary user authentication. Forexample, in the case where fingerprint recognition information isacquired from the primary user authentication, face recognitioninformation may be acquired from the secondary user authentication.

The secondary user authentication 650 may perform biometricauthentication using any one of fusion methods described above withreference to FIGS. 3A through 3D.

For example, the secondary user authentication may perform userauthentication by combining a comparison result acquired by comparingthe previously registered user information with first biometricinformation, which is carried out at the time of the primary userauthentication, with a comparison result acquired by comparing secondarybiometric information with the previously registered user information bya preset algorithm. Accordingly, the secondary user authentication 650may have a higher accuracy than the primary user authentication 610.

The security module 181 determines that the execution result of thesecondary user authentication 650 is either one of an authenticationsuccess or an authentication failure. Then, the secondary userauthentication 650 may end the user authentication.

On the other hand, the security module 181 may perform an authenticationinitialization operation when the execution of the secondary userauthentication 650 is completed. Accordingly, the user may retry theprimary user authentication again.

On the other hand, during the foregoing serial biometric authentication,user authentication is sequentially carried out according to a sequencedetermined based on a preset sequence or a specified criterion. At thistime, obtainable biometric information varies according to a surroundingenvironment at the time of performing biometric authentication, a user'scontext, and an operating state of the electronic device. At this time,the sensing accuracy of biometric information varies according to asurrounding environment at the time of performing biometricauthentication, a user's context, and an operating state of theelectronic device. However, in the related art, multimodal biometricauthentication has been carried out without considering such a context.

Furthermore, during the serial biometric authentication, subsequentbiometric authentication is carried out in accordance with a result ofpreceding biometric authentication. At this time, the subsequentbiometric authentication may be carried out when the result of thepreceding biometric authentication is no decision.

At this time, although the result of the preceding biometricauthentication should be determined as no decision, it may be determinedas an authentication failure according to the influence of a surroundingenvironment at the time of performing biometric authentication and thelike. Therefore, when a possibility in which the preceding biometricauthentication is determined as no decision increases due to theemphasis on a surrounding environment, the sensing accuracy of thepreceding biometric authentication decreases. On the contrary, when thesensing accuracy of the preceding biometric authentication isemphasized, a possibility in which the preceding biometricauthentication is determined as an authentication failure increases, andthus the subsequent biometric authentication itself is not carried out.As a result, the usability of the multimodal biometric authenticationdeteriorates.

Therefore, the present disclosure determines a variable criterionaccording to a surrounding environment or the like at the time ofperforming biometric authentication during serial biometricauthentication, thereby implementing a method of improving both theusability and the sensing accuracy of the multimodal biometricauthentication.

On the other hand, hereinafter, the preceding biometric authenticationis referred to as “primary user authentication” or “first userauthentication.” Furthermore, the subsequent biometric authentication isreferred to as “secondary user authentication” or “second userauthentication.” In addition, in the present disclosure, multimodalbiometric authentication in a serial manner has been described as anexample, but the present disclosure is not limited thereto.

For example, the present disclosure may be applicable to modified serialbiometric authentication in which secondary user authentication issequentially carried out even when the result of primary userauthentication is an authentication success according to a degree ofsecurity. Moreover, the present disclosure may also be applicable toparallel biometric authentication to the extent that does notcontradict.

FIGS. 8A through 8C illustrate a method of applying a variabledetermination criterion according to a context to perform serialbiometric authentication during multimodal biometric authenticationaccording to the present disclosure.

Referring to FIG. 8A, the security module 181 of the electronic device100 according to the present disclosure activates a biometricauthentication function for executing primary user authentication 810when an execution command for a function that requires multimodalbiometric authentication is applied. For example, the security module181 may execute a related program in the background of the electronicdevice 100 to execute a biometric authentication function in real timeor at predetermined intervals.

The security module 181 of the electronic device 100 may sense contextinformation (A) 820 at the time of or prior to sensing biometricinformation for primary user authentication 810.

Here, the context information (A) 820 may include surroundingenvironment information, operational state information of the electronicdevice, and user related information.

More specifically, the context information may include surroundingenvironment information such as ambient illuminance, ambient noise,ambient temperature, characteristic information of biometricinformation, movement information, and the like. Furthermore, thecontext information may include operational state information of theelectronic device 100 such as a posture of the electronic device 100,shake information, whether or not to execute a preset application, asecurity level of the executed application, a battery remaining amountof the electronic device 100, and the like. In addition, the contextinformation may include user related information such as inputinformation such as an input sequence of biometric information, userinformation, and the like.

Besides, various environmental sensors (e.g., an illuminance sensor, atemperature sensor, etc.), a gyro sensor, a geomagnetic sensor, and thelike provided in the electronic device 100 may be activated for thecollection of the context information. Moreover, for the collection ofthe context information, information stored in the memory 170 of theelectronic device 100 may be used.

In one embodiment, two or more pieces of the context information may beselected based on the characteristics of biometric information.Alternatively, context information indicating a context related tobiometric authentication of the electronic device 100 may be collected.Alternatively, the context information (A) 820 may be an element thatdetermines a biometric sensor to be used in the primary userauthentication 810.

Furthermore, in one embodiment, the context information (A) 820 that isonly relevant to the unique characteristics of biometric informationused for the primary user authentication 810 may be selected, but theirrelevant ones may not be selected. For example, when the primary userauthentication 810 is face authentication, the illuminance informationmay be selected but the noise information may be excluded.

The security module 181 of the electronic device 100 sets amultidimensional first variable determination criterion 830 includingthe context information 820 to authenticate biometric information sensedthrough biometric sensors during the primary user authentication 810.

Here, the biometric information may be classified according to the typeof biometric authentication. The biometric authentication includes faceauthentication, fingerprint authentication, voice authentication, irisauthentication, vein authentication, blood vessel authentication, andthe like, and exhibits a person's unique characteristics. Therefore, thebiometric information includes face recognition information, fingerprintrecognition information, voice recognition information, iris recognitioninformation, vein recognition information, blood vessel recognitioninformation, and the like. Therefore, biometric sensors for sensing thebiometric information denote sensors capable of sensing the user's facerecognition information, fingerprint recognition information, voicerecognition information, iris recognition information, vein recognitioninformation, blood vessel recognition information, and the like.

The security module 181 may perform multimodal biometric authenticationusing different biometric sensors according to the characteristicinformation of biometric information.

Here, the characteristic information of biometric information isinformation indicating the unique characteristic of biometricinformation. In addition, the unique characteristics may include qualityinformation of biometric information, spoofing information/anti-spoofinginformation of biometric information, characteristics related to amethod of collecting biometric information, and the like, as a whole.

For example, the quality information of face recognition information isa resolution of face recognition information. For example, the spoofinginformation/anti-spoofing information of face recognition information isinformation for allowing the face recognition information to recognize aface photo as an actual face, and the anti-spoofing information may bemovement sensing, eye blink detection, reflectivity information of a 2Dimage, thermal image information, heart rate information, frequencyspectrum information, audio and video synchronization information, andthe like. In addition, for example, characteristic information relatedto a method of collecting face recognition information may include acharacteristic of generating image capturing and a capturing sound foracquiring a face image, and a characteristic to be captured at apredetermined level or higher, and the like.

In addition, the first variable criterion may be determined inconsideration of biometric sensors that sense biometric information,collected context information, a matching score between the biometricinformation and previously registered user information, and a fake scoreof the biometric information as a whole.

Here, the previously registered user information denotes the features ofbiometric information previously registered with the electronic device100. For example, in the case of face authentication, a height of nose,a spacing between eyes, and a shape of eyebrow, which are the featuresof previously registered face recognition information, may be includedtherein as a whole. For example, in the case of vein authentication, ablood vessel/vein pattern of a user's palm, finger, hand, or the likemay be a feature of previously registered vein recognition information.

The matching score is a value obtained by converting the similaritiesbetween the feature points of the previously registered user informationand the biometric information detected by the biometric sensor into ascore. A genuine user has a high matching score. However, a matchingscore for the same user may vary depending on various contextinformation collected while sensing biometric information.

In addition, a fake relevance score of biometric information may includeat least one of an anti-spoofing score and a quality score.

The spoofing score is a score obtained by converting a possibility thatthe biometric information is fake information. Therefore, theanti-spoofing score is a score obtained by converting a possibility thatthe biometric information is not fake information. If the spoofing scoreis a fake score associated with a false acceptance rate (FAR), then theanti-spoofing score may be referred to as a fake score associated with afalse rejection rate (FRR).

The quality score is related to the detection of the characteristics ofbiometric information, and the quality of the detected biometricinformation is converted into a score.

For example, the security module 181 may convert a resolution of facerecognition information and a signal-to-noise ratio of voice recognitioninformation into a normalized quality score according to a presetcriterion. In addition, the security module 181 may performauthentication using biometric information having a higher score basedon the converted quality score. Here, various methods known in therelated art may be used for the conversion criterion of the normalizedquality score, and the detailed description thereof will be omittedhere.

The security module 181 may generate a variable decision function basedon the context information and the fake relevance score of biometricinformation. Here, the variable decision function may be generateddifferently each time biometric information is sensed by combining atleast one of the anti-spoofing score and the quality score with thecontext information.

The decision function is a function that determines whether a user whoenters biometric information is a genuine user or an impostor user. Thedecision function may be set to a specific threshold value, or may beset to a multidimensional function. For the decision function, theinitial value may be changed as the use of the biometric authenticationfunction of the electronic device 100 is accumulated, but the defaultvalue is set by the manufacturer.

However, in the present disclosure, a multidimensional function in whicha threshold value or a threshold range of no decision is varied eachtime is set as an authentication determination criterion inconsideration of various context information and a fake relevance scorecollected at the time of sensing biometric information. Therefore, adifferent decision function may be set for the same biometricinformation as well as for different biometric information.

Furthermore, the variable decision function is generated not only duringthe primary user authentication 810 but also during the secondary userauthentication 880. It is the same not only in the case of serialmultimodal biometric authentication as shown in FIG. 8A but also in thecase of parallel multimodal biometric authentication in which theprimary and secondary user authentications are simultaneously performed.

The generated variable decision function may be stored in the memory 170for each type of biometric information. In addition, the stored variabledecision function may be automatically updated upon recognition of thenext biometric information.

In this manner, every time the biometric information is sensed, thevariable decision function is determined to perform matching 840. Duringthe matching 840, a matching score is calculated by comparing biometricinformation used for the primary user authentication 810 with previouslyregistered user information. In addition, a final score may becalculated by combining the biometric fake relevance score (e.g.,anti-spoofing score, quality score, etc.) of biometric information withthe calculated matching score. Then, the variable decision functiongenerated based on the context information and the fake relevance scoreof biometric information is applied to the calculated final matchingscore to perform authentication.

According to this, the sensing accuracy of the primary userauthentication 810 is increased and the usability of the multimodalbiometric authentication through the secondary user authentication 880is also further improved.

As a result of performing the matching 840 of the primary userauthentication by applying the variable decision function, any one of anauthentication success 850, an authentication failure 860, and nodecision 870 is displayed as the authentication result.

The authentication result may be immediately displayed on the displayunit 151 of the electronic device 100. Alternatively, the authenticationresult may be displayed as a final authentication result obtained byfusing a plurality of authentication results after both the primary userauthentication 810 and the secondary user authentication 880 arecompleted.

In the present disclosure, if the authentication result of the primaryuser authentication 810 is the authentication success 850 or theauthentication failure 860, the secondary user authentication 880 maynot be performed. In other words, when the authentication result of theprimary user authentication 810 is no decision 870, the secondary userauthentication 880 may be carried out subsequently.

In the present disclosure, since the primary user authentication isperformed by applying a variable decision function generated byincluding the context information and the fake relevance information,the secondary user authentication 880 is not carried out due to theauthentication failure 860 of a genuine user, thereby solving a problemthat the usability of multimodal biometric authentication is lowered. Inaddition, a problem that the accuracy of sensing is lowered since thesecondary user authentication 880 is not performed due to theauthentication success 850 of an impostor user is also solved.

Even during the secondary user authentication 880, a multidimensionalsecond variable determination criterion is set 830′ by including thecontext information (B) 820′ collected while sensing biometricinformation. The setting of the second variable determination criterionis similar to the setting of the first variable determination criterion.

However, the context information (A) at the time of performing theprimary user authentication 810 is different from the contextinformation (B) at the time of performing the secondary userauthentication 880, and the respective fake relevance scores (e.g.,anti-spoofing score, quality score) are different, and thus a variabledecision function in the primary user authentication 810 and a variabledecision function in the secondary user authentication 880 are differentfrom each other.

During the matching 840′ of the secondary user authentication 880, asecond variable criterion 830′ is applied to the final score.Accordingly, an authentication success 850′ or an authentication failure860′ is displayed. At this time, a final authentication result fusedwith a result of the primary user authentication and a result of thesecondary user authentication may be displayed.

When the first and/or second user authentication result is displayed bythe security module 181, an operation related to the authenticationresult of the controller 180 of the electronic device 100 may becontrolled.

FIG. 8B illustrates a process of performing primary user authenticationin more detail, and FIG. 8C illustrates a process of performingsecondary user authentication in more detail.

Referring to FIG. 8B, an anti-scooping score 811, a quality score 812,and a matching score 813 are calculated based on the first biometricinformation sensed through the sensor (A). In addition, a variabledetermination criterion is set 815 in consideration of the anti-scoopingscore (SA), the quality score (QA) and the matching score (MA) relatedto the characteristics of the first biometric information as a whole, byincluding context information (CA) 814 collected at the time of sensingthe first biometric information. Furthermore, matching & decision 816 isthen carried out to apply the set variable determination criterion tothe final score.

Then, referring to FIG. 8C, an anti-scooping score 821, a quality score822, and a matching score 823 are calculated based on the secondbiometric information sensed through the sensor (B). At this time,factors of variable determination criteria (SA, QA, MA, CA) 815′ set inthe primary user authentication may be included as additional factors inthe setting 815′. In other words, the fusion 825 of the variabledetermination criteria may be carried out. Then, matching & decision 826that applies the fusion 825 of variable determination criteria to afinal score related to second biometric information is carried out.

Hereinafter, an authentication result according to a determinationcriterion in the related art and an authentication result according tothe application of a variable determination criterion according to thepresent disclosure will be described in comparison with each other withreference to FIGS. 9A through 9C.

FIG. 9A is an example of performing matching & decision according to afixed decision criterion in the related art when performing faceauthentication. A reference value/baseline (hereinafter referred to asan “upper threshold value” 901) at which the false acceptance rate (FAR)becomes zero is fixed to a normalized final score of 0.8. Furthermore, areference value/baseline (hereinafter referred to as a “lower thresholdvalue” 902) at which the false rejection rate (FRR) becomes zero isfixed to a normalized final score of 0.2.

When the final score of face authentication is equal to or greater than(or above) the upper threshold value 901, it becomes an authenticationsuccess, and when the final score of face authentication is less than(or below) the lower threshold value 902, it becomes an authenticationfailure. In addition, when it is between the upper threshold value 901and the lower threshold value 902, it becomes no decision.

However, depending on the context, it is required to increase ordecrease the upper threshold value 901 and/or the lower threshold value902. For example, when the ambient illumination is dark, for example, 3lux, a genuine user also fails authentication. In this case, when areduced lower threshold value 903 is applied to obtain an authenticationresult which is no decision other than an authentication failure, mostof the impostor will have no decision other than an authenticationfailure in the 3 lux environment, thereby greatly reducing the sensingaccuracy and the usability of primary user authentication.

However, in the present disclosure, as illustrated in FIG. 9B, avariable upper threshold value 910 and a variable lower threshold value920 are applied according to the level of ambient illuminance (e.g.,Gen, GenLow, BenDark). Here, the upper and lower threshold values may bedefined as a continuous value in a multidimensional function that variesaccording to the level of the context information, other than a specificvalue.

Thus, even if the scores are the same, it becomes no decision when theambient illuminance is dark, for example, at 3 lux, and it becomes anauthentication failure or an authentication success when the ambientilluminance is bright, and thus the accuracy of sensing improves.Moreover, when the ambient illuminance is dark, for example, 3 lux, itbecomes no decision and secondary user authentication is carried out,and thus the usability of multimodal biometric authentication withhigher security is further improved.

Referring to FIG. 9C, with respect to the matching & decision of thefinal score, at the first point (P1), it has the same result, that is,no decision, in both in a method of the related art and in a methodaccording to the present disclosure. However, at the second point (P2),since a variable determination criterion is applied to the ambientilluminance by reflecting dark environment information, it becomes nodecision in the present disclosure. However, in the method of therelated art, it becomes an authentication failure at the second point(P2).

Furthermore, in the case where the ambient illuminance is normal at thethird point (P3), a variable decision function in consideration of otherfactors than the illuminance information, for example, quality scoressuch as resolution, flow, distortion, and the like, and anti-spoofingscores such as 2D image reflectivity information, thermal imageinformation, and the like as a whole is applied. Accordingly, it isdetermined as an authentication success in the present disclosure,whereas it becomes no decision according to a fixed determinationcriterion in the related art. In the case where the result of thesecondary user authentication is fused to output the finalauthentication result even though the primary user authentication is anauthentication success, the determination of the authentication successat the third point (P3) contributes to enhance the sensing accuracy.

As described above, according to the present disclosure, a variabledecision function is generated as a multidimensional function in whichan upper threshold value, a lower limit threshold value related to thedetermination of user authentication is varied according to a level ofthe context information, and/or a fake relevance score of biometricinformation (e.g., quality score, spoofing/anti-spoofing score), andapplied to the final score.

Here, the upper threshold value is a value or a line at which the falseacceptance rate (FAR) becomes zero, and the lower threshold value is avalue or a line at which the false rejection rate (FRR) becomes zero. Inaddition, at least one of the upper threshold value or the lowerthreshold value is increased or decreased according to factors such as alevel of the context information, and/or a fake relevance score of thebiometric information.

On the other hand, when the environment information is accelerationinformation or noise information, a variable decision function in asimilar form to the graph shown in FIG. 9B, that is, a variableupper/lower threshold value, may be generated. However, when theenvironment information is related to vein recognition information orrelated to iris recognition information, a variable decision function inan opposite form to the graph shown in FIG. 9B, that is, a variableupper/lower threshold value, may be generated.

FIG. 10 is a flowchart of a process of applying a variable determinationcriterion according to a context to perform serial biometricauthentication during multimodal biometric authentication according tothe present disclosure.

When a biometric authentication function is activated, the electronicdevice 100 according to the present disclosure may sense first biometricinformation through a first biometric sensor, and sense contextinformation at the time of sensing/recognizing the first biometricinformation (S10). Next, a multidimensional variable determinationcriterion for the first biometric information is set by including thesensed context information (S20). Then, first user authentication forthe first biometric information is carried out based on the setmultidimensional variable determination criterion (S30). Then, seconduser authentication using second biometric information is carried outaccording to the execution result of the first user authentication(S40).

FIGS. 11A through 14 are examples showing specific multimodal biometricauthentication elements and context information to which a variabledetermination criterion according to the present disclosure is applied.

According to an embodiment of the present disclosure, the second userauthentication is carried out when the authentication result of thefirst user authentication is no decision that does not correspond toboth an authentication success and an authentication failure.

Moreover, in one embodiment, the second user authentication may combinea comparison result obtained by comparing previously registered userinformation and first biometric information performed at the time offirst user authentication and a comparison result obtained by comparingsecond biometric information and previously registered user informationto perform final user authentication.

First, in FIG. 11A, face authentication 1101 is used as the first userauthentication, and illuminance information (A) 1102 is selected as thecontext information. Furthermore, a multidimensional first variabledetermination criterion 1104 is determined in consideration of ananti-spoofing score 1103, a quality score 1104 and a matching score 1105related to face recognition information, including the collectedilluminance information (A). In addition, applying the first variabledetermination criterion to the final score results in any one result ofan authentication success 1106, an authentication failure 1107, and nodecision 1108.

Referring to FIG. 11B, when the authentication result of the faceauthentication 1101 is no decision 1108 in FIG. 11A, second userauthentication 1111 fused with face authentication and fingerprintauthentication is carried out. At this time, acceleration information(B) 1112 may be selected as the context information. At this time, theacceleration information (B) 1112 may be different from accelerationinformation collected at the time of the face authentication 1101. Next,a multidimensional second variable determination criterion is determinedby further considering an anti-spoofing score 1114 related fingerprintrecognition information, a quality score 1115, a matching score 1116,and a first variable determination criterion value 1113 determinedduring the face authentication, including the acceleration information(B) 1112.

In addition, applying the second variable determination criterion to thefinal score results in either one result of an authentication success1117 and an authentication failure 1116.

FIG. 12 is a flowchart showing a method of determining a variabledetermination criterion for multimodal biometric authentication withreference to FIGS. 11A and 11B.

When multimodal biometric authentication function according to thepresent disclosure is activated, a first variable criterion is set byincluding first context information related to the first userauthentication as shown in FIG. 12 (S1210). Then, it is determinedwhether the result of the first user authentication is no decisionaccording to the set first variable determination criterion (S1220).When it is a section other than no decision, it is determined as anauthentication success or an authentication failure (S1230).

Here, the no decision is a case where a final score in consideration ofa matching score related to the first user authentication, a fakerelevance score of the biometric information of the first userauthentication, and the context information as a whole is less than theforegoing lower threshold value. In the present disclosure, the lowerthreshold value varies according to the context information and/or thefake relevance score

When the result of the first user authentication is no decision, asecond variable determination criterion including second contextinformation related to the second user authentication is set inconsideration of the first variable determination criterion (S1240).Then, the second user authentication is carried out based according tothe set second variable determination criterion (S1250).

Meanwhile, in one embodiment, both the first and second biometricinformation may be used while performing the second user authentication.Furthermore, a multidimensional variable determination criterion duringthe second user authentication may be determined by further consideringa multidimensional variable determination criterion used for the firstuser authentication.

FIG. 13 shows a process of performing serial multimodal biometricauthentication according to the present disclosure using voiceauthentication and iris authentication as multimodal biometricauthentication. During the first user authentication, a multidimensionalvariable determination criterion including noise information (A) is set.When the result of voice authentication is no decision, voiceauthentication and iris authentication are fused, and a multidimensionalvariable determination criterion including illuminance information (B)is set again.

FIG. 14 shows a process of performing serial multimodal biometricauthentication according to the present disclosure using faceauthentication and vein authentication as multimodal biometricauthentication. During the first user authentication, a multidimensionalvariable determination criterion including illuminance information (A)is set. When the result of face authentication is no decision, faceauthentication and vein authentication are fused, and a multidimensionalvariable determination criterion including illuminance information (A′)is set again.

Furthermore, in one embodiment, biometric information and contextinformation for the second user authentication may be selected based onthe biometric information and context information used for the firstuser authentication when performing the second user authentication. Forexample, when the illuminance information (A) collected in the faceauthentication is dark, it is seen that vein authentication that can beauthenticated in a dark environment is selected, and the illuminanceinformation, which is context information related thereto, is selectedagain.

However, the illuminance information (A′) collected during veinauthentication differs from the illuminance information (A) collectedduring face authentication, and acts as a different factor when settinga variable determination criterion. For example, during the faceauthentication, as a level of the illuminance information (A′)increases, a variable decision function similar to that of FIG. 9B isgenerated. On the contrary since the vein authentication uses infraredrays to project the user's blood vessel, a variable decision function inan opposite form to that of FIG. 9B will be generated as the level ofthe illuminance information (A′) increases.

FIG. 15 is a conceptual view illustrating a method of applying avariable determination criterion according to a context and fusing aplurality of sensors, context information, and the variabledetermination criterion to perform parallel biometric authenticationduring multimodal biometric authentication according to the presentdisclosure.

FIG. 15 is an example of applying a variable determination criterionaccording to the present disclosure to perform authentication duringparallel multimodal biometric authentication in which first userauthentication using the sensor (A) and second user authentication usingthe sensor (B) are carried out at the same time.

In FIG. 15, first biometric information and second biometric informationmay be sensed at the same time by the sensor (A) and the sensor (B). Atthis time, at the time of calculating the matching score of the firstbiometric information, the fused context information, that is, both thecontext information (A) and the context information (B), may be takeninto consideration. Furthermore, at the time of calculating the matchingscore of the second biometric information, the fused contextinformation, that is, both the context information (A) and the contextinformation (B), may be taken into consideration.

In addition, a first variable determination criterion 1510 applied tothe final score of the first biometric information may also beapplicable to the final score of the second biometric information.Furthermore, a second variable determination criterion 1520 applied tothe final score of the second biometric information may also beapplicable to the final score of the first biometric information.Alternatively, a third variable determination criterion fused with thefirst and second variable criteria may also be applicable to the finalscore of the first biometric information and the final score of thesecond biometric information, respectively.

Moreover, the authentication result of the first biometric informationand the authentication result of the second biometric information arefused 1530 to calculate the final authentication result.

As described above, an electronic device according to the presentdisclosure may determine a variable determination criterion inconsideration of a fake relevance score related to a surroundingenvironment at the time of performing biometric authentication and thecharacteristics of biometric authentication to perform biometricauthentication, thereby improving all the usability, convenience andsensing accuracy of biometric authentication. In addition, whether ornot to perform secondary authentication may be determined according tothe execution result of primary authentication, thereby improving theauthentication speed of biometric authentication. Furthermore, amultidimensional function in which a threshold value or a thresholdrange of no decision is varied each time may be applied toauthentication in consideration of various context information collectedwhile sensing biometric information and a fake relevance score as awhole, thereby improving the accuracy of biometric authentication duringsecondary authentication.

2363 The foregoing present disclosure may be implemented as codesreadable by a computer on a medium written by the program. Thecomputer-readable media includes all types of recording devices in whichdata readable by a computer system can be stored. Examples of thecomputer-readable media may include ROM, RAM, CD-ROM, magnetic tape,floppy disk, and optical data storage device, and the like, and alsoinclude a device implemented in the form of a carrier wave (for example,transmission via the Internet). In addition, the computer may includethe controller 180 of the electronic device. The foregoing embodimentsare merely exemplary and are not to be considered as limiting thepresent disclosure. The scope of the invention should be determined byreasonable interpretation of the appended claims and all changes thatcome within the equivalent scope of the invention are included in thescope of the invention.

What is claimed is:
 1. An electronic device for performing multimodalbiometric authentication, the electronic device comprising: a firstbiometric sensor configured to sense first biometric information; asecond biometric sensor configured to sense second biometricinformation; a security module configured to set a multidimensionalvariable determination criterion including context information at thetime of sensing at least one of the first biometric information and thesecond biometric information, and perform first user authentication byapplying the multidimensional variable determination criterion to amatching score for the first biometric information or the secondbiometric information, and determine whether or not to perform seconduser authentication using the remaining biometric information based on aresult of the first user authentication; and a controller configured tocontrol an operation of the electronic device based on at least one ofthe first and second user authentication results carried out by thesecurity module.
 2. The electronic device of claim 1, wherein themultidimensional variable determination criterion is determined inconsideration of biometric sensors for sensing biometric information,the context information, the matching score, and a fake relevance scoreof the biometric information as a whole.
 3. The electronic device ofclaim 1, wherein the context information comprises at least one ofambient illuminance, ambient noise, ambient temperature, characteristicinformation of biometric information, movement information, a posture ofthe electronic device, an execution of a preset application, a batteryremaining amount of the electronic device, an input sequence ofbiometric information, and user information, and at least one of thecontext information is selected based on the characteristics ofbiometric information.
 4. The electronic device of claim 3, wherein thesecurity module calculates a matching score by comparing the firstbiometric information or the second biometric information withpreviously registered user information, calculates a final score bycombining the calculated matching score with a fake relevance score ofthe biometric information, and performs user authentication by applyinga variable decision function generated on the basis of the contextinformation and the fake relevance score of the biometric information tothe calculated final matching score.
 5. The electronic device of claim4, wherein the fake relevance score of the biometric informationcomprises at least one of an anti-spoofing score and a quality score,and the variable decision function is generated by combining at leastone of the anti-spoofing score and the quality score with the contextinformation.
 6. The electronic device of claim 4, wherein the variabledecision function is generated such that an upper threshold value and alower threshold value related to the user authentication determinationvary according to a level of the context information.
 7. The electronicdevice of claim 6, wherein the upper threshold value is a value at whicha false acceptance rate (FAR) becomes zero, the lower threshold value isa value at which a false rejection rate (FRR) becomes zero, and at leastone of the upper threshold value and the lower threshold value isincreased or decreased according to a level of the context information.8. The electronic device of claim 6, wherein the second userauthentication is carried out when the authentication result of thefirst user authentication does not correspond to both an authenticationsuccess and an authentication failure.
 9. The electronic device of claim8, wherein the no decision is a case where a final score inconsideration of the matching score, the fake relevance score of thebiometric information, and the context information as a whole is lessthan the lower threshold value, and the lower threshold value is variedaccording to the context information.
 10. The electronic device of claim1, wherein when performing the second user authentication, biometricinformation and context information for the second user authenticationare selected based on biometric information and context information usedfor the first user authentication.
 11. The electronic device of claim 1,wherein when performing the second user authentication, both the firstand second biometric information are used, and a multidimensionalvariable determination criterion during the second user authenticationis determined by further considering a multidimensional variabledetermination criterion used for the first user authentication.
 12. Theelectronic device of claim 11, wherein when performing the second userauthentication, a multidimensional variable determination criterionduring the second user authentication is determined by furtherconsidering both a multidimensional variable determination criterionused for the first user authentication and context information used forthe first user authentication.
 13. A method of operating an electronicdevice that performs multimodal biometric authentication, the methodcomprising: sensing biometric information of at least one of a firstbiometric sensor and a second biometric sensor; setting amultidimensional variable determination criterion including contextinformation at the time of sensing at least one of biometric informationof the first and second biometric sensors; performing first userauthentication by applying the multidimensional variable determinationcriterion to a matching score for the at least one biometricinformation; determining whether or not to perform second userauthentication using the remaining biometric information based on aresult of the first user authentication; and controlling an operation ofthe electronic device based on at least one of the first and second userauthentication results.
 14. The method of claim 13, wherein said settinga multidimensional variable determination criterion is setting inconsideration of biometric sensors for sensing biometric information,the context information, the matching score, and a fake relevance scoreof the biometric information as a whole.
 15. The method of claim 14,wherein said performing first user authentication comprises: calculatinga matching score by comparing the first biometric information or thesecond biometric information with previously registered userinformation; calculating a final score by combining the calculatedmatching score with a fake relevance score of biometric information; andperforming the first user authentication by applying a variable decisionfunction generated based on the context information and the fakerelevance score of biometric information to the calculated final score.16. The method of claim 15, wherein the variable decision function isgenerated such that an upper threshold value and a lower threshold valuerelated to the user authentication determination vary according to alevel of the context information, and the second user authentication iscarried out when the final score corresponds to between variable upperand a lower threshold values as a result of performing the first userauthentication.